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Sir: 

In conjunction with the filing of the present application, and prior to an initial Official Action 
on this matter, please amend the above-identified application as follows: 

IN THE SPECIFICATION 
In Paragraph [0005], please substitute the paragraph as follows: 

Currently, the prior-art methods to capture omni-directional images may be categorized as below: 

(a) a method to cover about hemispheric viewing angle using non-planer mirror and 

a camera; 

(b) a method to cover wider viewing angle using multiple planer mirror and multiple 

cameras; 

(c) a method to generate apanoramic image by connecting aplurality of images which 
are obtained as a camera rotates; 

(d) a method to produce a single image as by coimecting a plurality of images which 
are obtained as a line-scan camera rotates (http://www.e-pan.com/index.html); 

(e) a method to cover wider viewing angle by a plurality of imaged which are 
obtained as a plurality of cameras each of which covers different viewing angle(for example, US 
patent 5,023,725); and 

(f) a method to cover about hemispheric viewing angle using a camera with fisheye 
lens (for example, US patent 5,185,667). 

In Paragraph [0026], please substitute the paragraph as follows: 



The accompanying drawings, which are included to provide a further understanding of the invention 
and are incorporated in and constitute a part of this appHcation, illustrate exemplary embodiment(s) 
of the invention and together with the description serve to explain the principle of the invention. 

In Paragraph [0103], please substitute the paragraph as follows: 

FIG. 1 5 is a flow diagram illustrating a process of acquiring 3 -dimensional depth data using epipolar 
geometry by the omni-directional 3 -dimension image data acquisition apparatus according to the 
invention. Namely, FIG. 1 5 shows steps of image data acquisition to 3-dimensional data extraction 
using the spherical coordinate arrangement and applymg epipolar geometry. This method comprising 
the steps of: 

assuming a spherical coordinate where the center of spherical coordinate is set to the 
projection center of the camera 1 1 and axis of the spherical coordinate and optical axis of the camera 
11 is coUinearSlOO; 

finding a mapping table or function between points on the spherical coordinate and 
points on the image plane of the camera 11 by either empirical or mathematical calibration S200; 

acquiring two images of same scene with small displacement between two 2D images 
acquisition locations S3 00; 

mapping those 2D images to the surfaces of two spheres (or to some data structure 
to express spheres) using the mapping table or function S400; 

rotating said spherical coordinates of those spheres to make their axes and connecting 
line between optical projection centers of paired cameras 1 1 become coUinear S500; 

applying epipolar geometry to correlate points between said two images S600; 

determing spatial location of correlated points by triangulation S700; and 

possibly repeating the steps fi-om S300 to S700 to make more images to possibly 

acquire. 

IN THE CLAIMS 

In Claim 1, the claim as follows: 

1 . (Amended) An omni-directional image and 3-Dimensional data acquisition apparatus, comprising: 
a multi-camera module constructed in a manner [that] comprising a plurality of 
cameras that are symmetrically arranged with a specific point in a plane, each of the cameras taking 
charge of each of divided angles such that the camera module can take an omni-directional 
continuous panoramic photograph of surrounding objects with the specific point; 

first frame grabbers each of which is electrically connected to each of the cameras of 
the multi-camera module, to grab photographed images by frames; 

an exposure calculator electrically coimected to the first frame grabbers, to calculate 
exposure of each camera, based on the grabbed images by frames; 



an exposure signal generator electrically connected to each camera, to transmit 
information about the exposure as a signal on the basis of the exposure calculated by the exposure 
calculator; 

storage means electrically connected to the each first fi-ame grabber, to store images 
photographed by the cameras according to photographing location and photographing time; 

a GPS sensor to sense the photographing location and photographing time; 

an annotation entering unit electrically connected to the GPS sensor to calculate 
location and time corresponding to each frame based on sensed data of the GPS sensor, the 
annotation entering unit being electrically coimected to the storage means to enter the calculated 
location and time in each frame as annotation; and 

a trigger signal generator electrically connected the storage means, the exposure signal 
generator, the annotation entering unit, the trigger signal generator selectively transmits a trigger 
signal to the exposure signal generator or the annotation entering unit in order that the cameras start 
to photograph the objects according to the trigger signal; and wherein the multi-camera module are 
vertically stacked and formed in at least two layers in the direction of height. 

In Claim 3, please substitute the claim as follows: 

3. (Amended) The apparatus as claimed in claim 1, wherein the storage means is one of digital 
storage devices comprised of a hard disk, compact disk, magnetic tape and memory. 

In Claim 6, please substitute the claim as follows: 

6. (Amended) The apparatus as claimed in claim 1, wherein the multi-camera module fiirther 
comprises at least one camera placed at the top thereof so that the camera can photograph an object 
upward. 

In Claim 10, please substitute the claim as follows: 

10. (Amended) A method for extending dynamic range of images, comprising the steps of: 

acquiring for multi images of an object, the multi images being photographed by the 
cameras which have different exposure amount each other, wherein the multi camera module 
comprising a plurality of cameras which are symmetrically arranged at a specific point in a plane, 
and which take charge of each allocating viewing angle calculated by 360° divided by the number 
of the cameras; 

selectively extracting for regions in the multi images, wherein the regions have 
constant exposure amount; and 

acquiring for images of dynamic range extension, which are generated by composing 
the extracting regions. 

In Claim 11, please substitute the claim as follows: 



11. (Amended) An omni-directional image and 3 -Dimensional data acquisition apparatus, 
comprising: 

a multi-camera module constructed in a manner comprised of a plurality of cameras 
that are symmetrically arranged with a specific point in a plane, each of the cameras taking charge 
of each of divided angles such that the camera module can take an omni-directional continuous 
panoramic photograph of surrounding objects with the specific point; 

an elevator for elevating the multi-camera module vertically; 

first frame grabbers each of which is electrically connected to each of the cameras of 
the multi-camera module, to grab photographed images by frames; 

an exposure calculator electrically connected to the first frame grabbers, to calculate 
exposure of each camera, based on the grabbed images by frames; 

an exposure signal generator electrically connected to each camera, to transmit 
information about the exposure as a signal on the basis of the exposure calculated by the exposure 
calculator; 

storage means electrically connected to the each first fir^e grabber, to store images 
photographed by the cameras according to photographing location and photographing time; 

a GPS sensor to sense the photographing location and photographing time; 

an annotation entering unit electrically connected to the GPS sensor to calculate 
location and time corresponding to each frame based on sensed data of the GPS sensor, the 
annotation entering xmit being electrically connected to the storage means to enter the calculated 
location and time in each frame as annotation; and 

a trigger signal generator electrically coimected the storage means, the exposure signal 
generator, the annotation entering unit, the trigger signal generator selectively transmits a trigger 
signal to the exposure signal generator or the annotation entering unit in order that the cameras start 
to photograph the objects according to the trigger signal 

IN THE ABSTRACT 

On page 31, please substitute the paragraph as follows: 

A method and apparatus for omni-directional image and 3 -dimensional data acquisition with data 
annotation and dynamic rage extension method is capable of omni-directionally photographing, 
acquiring 3 -dimensional images photographed by cameras having each different exposure amount 
in connection with the direction of height of an object, extending dynamic range, and generating an 
geographical information by entering an annotation such as photographing location and time into the 
photographed images, which can be connected with other geographical information system database. 
The apparatus includes one or more multi camera module(s) which are stacked and formed multi 
layers in the direction of height for acquiring 3-dimensional images and extending dynamic range 
of the 3-dimensional images, wherein each multi camera module includes a plurality of cameras 
symmetrically arranged with a specific point in a plane. 
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REMARKS ON PRELIMINARY AMENDMENT 

Assistant Commissioner for Patents 
Washington, D.C. 20231 

Sir: 

In this preliminary amendment, please consider the following remarks in conjimction with 
the amendments to the above-identified application as follows: 

REMARKS 

The present Preliminary Amendment has been entered for the purpose of placing the 
application into a more proper U.S. format. In particular, certain grammatical and idiomatic 
inconsistencies have been corrected by amendment to the specification, and the application is 
corrected for certain typographical errors found in the originally submitted application. No new 
matter has been added by these amendments. 

The claims and Abstract have been amended so as to conform with U.S. requirements. 

Applicant respectfully requests that the present Amendment be entered prior to an initial 
Official Action on the present application. 

Respectfully submitted. 

Date John S. Egbert 

Reg. No. 30,627 
Andrew W. Chu 
Reg. No. 46,625 
Attorney for Applicant 
Harrison & Egbert 
412 Main Street, T Floor 
Houston, Texas 77002 
(713)224-8080 
(713)223-4873 fax 
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Assistant Commissioner for Patents 
Washington, D.C. 20231 

Sir: 

In conjunction with the fiUng of the present application, and prior to an initial Official Action 
on this matter, please amend the above-identified application as follows: 

IN THE SPECIFICATION 

In Paragraph [0005], the paragraph has been amended as follows: 

Currently, the prior-art methods to capture omni-directional images may be categorized as below: 

(a) a method to cover about hemispheric viewing angle using non-planer mirror and 

a camera; 

(b) a method to cover wider viewing angle using multiple planer mirror and multiple 

cameras; 

(c) a method to generate a panoramic image by connecting a plurality of images which 
are obtained as a camera rotates; 

(d) a method to produce a single image as by connecting a plurality of images which 
are obtained as a line-scan camera rotates (http://www.e-pan.com/index.html); 

{e} a method to cover wider viewing angle by a plurality of imaged which are 
obtained as a plurality of cameras each of which covers different viewing angle(for example, US 
patent 5,023,725); and 

(f) a method to cover about hemispheric viewing angle using a camera with fisheye 
lens (for example, US patent 5,185,667). 

In Paragraph [0026], the paragraph has been amended as follows: 



The accompanying drawings, which are included to provide a further understanding of the invention 
and are incorporated in and constitute a part of this appHcation, illustrate exemplary embodiment(s) 
of the invention and together with the description serve to explain the principle of the invention. [In 
the drawings;] 

In P£u-agraph [0103], the paragraph has been amended as follows: 

FIG. 1 5 is a flow diagram illustrating a process of acquiring 3 -dimensional depth data using epipolar 
geometry by the omni-directional 3 -dimension image data acquisition apparatus according to the 
invention. Namely, FIG. 15 shows steps of image data acquisition to 3-dimensional data extraction 
using the spherical coordinate arrangement and applying epipolar geometry. This method comprising 
the steps of: 

assuming a spherical coordinate [is assumed] where the center of spherical coordinate 
is set to the projection center of the camera 1 1 and axis of the spherical coordinate and optical axis 
of the camera 1 1 is collinearS100[.]; 

[Mapping] finding a mapping table or function between points on the spherical 
coordinate and points on the image plane of the camera 1 1 [is found] by either empirical or 
mathematical calibration S200[.]i 

[Acquire] acquiring two images of same scene with small displacement between two 
2D images acquisition locations S300[.]i 

[Map] mapping those 2D images to the surfaces of two spheres (or to some data 
structure to express spheres) using the mapping table or function S400[.]; 

[Rotate the] rotating said spherical coordinates of those spheres to make their axes 
and connecting line between optical projection centers of paired cameras 11 become collinear 
S500[.]; 

[Apply] applvmg epipolar geometry to correlate points between said two images 

S600[.]i 

[Determine] determing spatial location of correlated points by triangulation S700[.]i 

and 

[Further, the repeat of] possibly repeating the steps from S300 to S700 [makes] to 
make more images to possibly acquire. 

IN THE CLAIMS 

In Claim 1, the claim has been amended as follows: 

1 . (Amended) An omni-directional image and 3-Dimensional data acquisition apparatus, comprising: 
a multi-camera module constructed in a manner [that] comprising a plurality of 
cameras tiiat are symmetrically arranged with a specific point in a plane, each of the cameras taking 
charge of each of divided angles such that the camera module can take an omni-directional 
continuous panoramic photograph of surrounding objects with the specific point; 

first frame grabbers each of which is electrically connected to each of the cameras 

of the multi-camera module, to grab photographed images by frames; 



an exposure calculator electrically connected to the first frame grabbers, to calculate 
exposure of each camera, based on the grabbed images by frames; 

an exposure signal generator electrically connected to each camera, to transmit 

information about the exposure as a signal on the basis of the exposure calculated by the exposure 
calculator; 

storage means electrically connected to the each first frame grabber, to store images 

photographed by the cameras according to photographing location and photographing time; 

a GPS sensor to sense the photographing location and photographing time; 

an annotation entering unit electrically connected to the GPS sensor to calculate 
location and time corresponding to each frame based on sensed data of the GPS sensor, the 
armotation entering unit being electrically connected to the storage means to enter the calculated 
location and time in each frame as armotation; and 

a trigger signal generator electrically connected the storage means, the exposure signal 
generator, the annotation entering unit, the trigger signal generator selectively transmits a trigger 
signal to the exposure signal generator or the annotation entering unit in order that the cameras start 
to photograph the objects according to the trigger signal; and [ 

Jwherein the multi-camera module are vertically stacked and formed in at least two layers in the 
H= direction of height. 

In Claim 3, the claim has been amended as follows: 

y 

Jl 3. (Amended) The apparatus as claimed in claim 1, wherein the storage means is one of digital 
^ storage devices [including] comprised of a hard disk, compact disk, magnetic tape and memory. 

1.11 

In Claim 6, the claim has been amended as follows: 

IS^ 6. (Amended) The apparatus as claimed in claim 1 , wherein the multi-camera module further [has] 
S| comprises at least one camera placed at the top thereof so that the camera can photograph an object 
|2j upward. 

f IJ . 

In Claim 10, the claim has been amended as follows: 

10, (Amended) A method for extending dynamic range of images, comprising the steps of: 

acquiring for multi images of an object, the multi images [are] being photographed 

by the cameras which have different exposure amount each other, wherein the multi camera module 

[including] comprising a plurality of cameras which are symmetrically arranged at a specific point 

in a plane, and which take charge of each allocating viewing angle calculated by 360"^ divided by the 

number of the cameras; 

selectively extracting for regions in the multi images, wherein the regions have 

constant exposure amount; and 

acquiring for images of dynamic range extension, which are generated by composing 

the extracting regions. 
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In Claim 1 1, the claim has been amended as follows: 

11. (Amended) An omni-directional image and 3 -Dimensional data acquisition apparatus, 
comprising: 

a multi-camera module constructed in a manner [that] comprised of a plurality of 
cameras tiiat are symmetrically arranged with a specific point in a plane, each of the cameras taking 
charge of each of divided angles such that the camera module can take an omni-directional 
continuous panoramic photograph of surrounding objects with the specific point; 
an elevator for elevating the multi-camera module vertically; 

first jframe grabbers each of which is electrically connected to each of the cameras of 
the multi-camera module, to grab photographed images by frames; 

an exposure calculator electrically connected to the first frame grabbers, to calculate 
exposure of each camera, based on the grabbed images by frames; 

an exposure signal generator electrically connected to each camera, to transmit 
information about the exposure as a signal on the basis of the exposure calculated by the exposure 
calculator; 

storage means electrically connected to the each first frame grabber, to store images 
photographed by the cameras according to photographing location and photographing time; 
a GPS sensor to sense the photographing location and photographing time; 

an annotation entering unit electrically connected to the GPS sensor to calculate 
location and time corresponding to each frame based on sensed data of the GPS sensor, the 
annotation entering unit being electrically connected to the storage means to enter the calculated 
location and time in each frame as annotation; and 

a trigger signal generator electrically connected the storage means, the exposure signal 
generator, the annotation entering unit, the trigger signal generator selectively transmits a trigger 
signal to the exposure signal generator or the annotation entering unit in order that the cameras start 
to photograph the objects according to the trigger signal. 

IN THE ABSTRACT 

On page 3 1 , the paragraph has been amended as follows: 

A method and apparatus for omni-directional image and 3 -dimensional data acquisition with data 
annotation and dynamic rage extension method is capable of omni-directionally photographing, 
acquiring 3 -dimensional images photographed by cameras having each different exposure amount 
in connection with the direction of height of an object, extending dynamic range, and generating an 
geographical information by entering an annotation such as photographing location and time into the 
photographed images, which can be connected with other geographical information system database. 
The apparatus [comprises] includes one or more multi camera module(s) which are stacked and 
formed multi layers in the direction of height for acquiring 3-dimensional images and extending 
dynamic range of the 3-dimensional images, wherein each multi camera module includes a plurality 
of cameras symmetrically arranged with a specific point in a plane. [Further, the multi camera 
modules of the apparatus is connected to a computer vision system in order to control of 
photographing and to store the photographed images, and is capable of mounting on a mobile 
means.] 



